Add compute_shared_memory_aggs used by shared memory groupby #17162
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PointKernel
added
non-breaking
PointKernel
commented
Oct 24, 2024
PointKernel
commented
Oct 24, 2024
davidwendt
reviewed
Oct 25, 2024
davidwendt
reviewed
Oct 25, 2024
davidwendt
reviewed
Oct 25, 2024
Co-authored-by: David Wendt <[email protected]>
ttnghia
reviewed
Oct 28, 2024
ttnghia
reviewed
Oct 28, 2024
3 tasks
|
@davidwendt would you please take another pass over this PR? |
davidwendt
reviewed
Oct 30, 2024
davidwendt
reviewed
Oct 30, 2024
davidwendt
approved these changes
Oct 30, 2024
ttnghia
reviewed
Oct 30, 2024
| @@ -23,6 +23,91 @@ | |||
| #include <cuco/static_set_ref.cuh> | |||
|
|
|||
| namespace cudf::groupby::detail::hash { | |||
| // TODO: TO BE REMOVED issue tracked via #17171 | |||
There was a problem hiding this comment.
Yes, #17171 should be addressed to avoid code duplication.
ttnghia
reviewed
Oct 30, 2024
ttnghia
approved these changes
Oct 30, 2024
Co-authored-by: David Wendt <[email protected]> Co-authored-by: Nghia Truong <[email protected]>
bdice
approved these changes
Oct 30, 2024
Co-authored-by: Bradley Dice <[email protected]>
|
/merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This work is part of splitting the original bulk shared memory groupby PR #16619.
This PR introduces the
compute_shared_memory_aggsAPI, which is utilized by the shared memory groupby. The shared memory groupby process consists of two main steps. The first step was introduced in #17147, and this PR implements the second step, where the actual aggregations are performed based on the offsets from the first step. Each thread block is designed to handle up to 128 unique keys. If this limit is exceeded, there won't be enough space to store temporary aggregation results in shared memory, so a flag is set to indicate that follow-up global memory aggregations are needed to complete the remaining aggregation requests.Checklist